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The Influence of Tech-Sawyness and Clicker Use on Student Learning 


Abstract 

The purpose of the study was to investigate the influence of technology proficiency and clicker use on 
students’ perceptions of clickers, engagement and class grade point average. Four hundred five students 
completed a questionnaire that measured Student Technology Proficiency (STP; Garcia and Zapf, in press), 
and participated in the validation of two new dependent measures: perception of clickers and student 
engagement. Class GPA was collected after the semester ended. A 2x2 MANOVA experiment was conducted 
and yielded no differences between students enrolled in a clicker class versus not, but there were several 
findings between STP groups. Students high in STP had a favorable view of clickers compared to students low 
in STP, but students low in STP had higher engagement and earned higher grades than students high in STP. 
We speculate that clickers continue to fill a pedagogical niche, but only in conjunction with effective teaching 
practices. Implications for teaching and learning are discussed. 
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Abstract 

The purpose of the study was to investigate the influence of technology proficiency and 
clicker use on students' perceptions of clickers, engagement and class grade point average. 
Four hundred five students completed a questionnaire that measured Student Technology 
Proficiency (STP; Garcia and Zapf, in press), and participated in the validation of two new 
dependent measures: perception of clickers and student engagement. Class GPA was 
collected after the semester ended. A 2x2 MANOVA experiment was conducted and yielded 
no differences between students enrolled in a clicker class versus not, but there were 
several findings between STP groups. Students high in STP had a favorable view of clickers 
compared to students low in STP, but students low in STP had higher engagement and 
earned higher grades than students high in STP. We speculate that clickers continue to fill 
a pedagogical niche, but only in conjunction with effective teaching practices. Implications 
for teaching and learning are discussed. 

Keywords: student response systems, clickers, student technology proficiency, 
engagement, academic performance 


Introduction 

Student response systems, commonly called clickers, are a relatively new method for 
engaging students in the college classroom. These cell-phone sized devices have a keypad 
and use interactive technology that allows students to answer questions posted on 
PowerPoint slides. Immediate feedback can be given once all students have answered the 
question in the form of colorful bar graphs, providing a practical way to inform both the 
students and instructor whether the students comprehend course content. Although much 
of the recent literature on the use of clickers in classrooms has focused on the pedagogical 
purposes of this technology (i.e. Beekes, 2006; Cleary, 2008; Kaleta & Joosten, 2007; 
Morling, McAuliffe, Coh, & DiLorenzo, 2008; Trees & Jackson, 2007; Wit, 2003), little is 
known about whether their appeal is related to our society's acceptance of technology in 
general. In a culture where educational institutions continue to stress information literacy 
and technology use, it could be that clicker popularity among students and instructors is 
related to our cultural familiarity with technology, rather than solely because clickers help 
students learn. Previous studies have not separated technology familiarity from the appeal 
of clickers and their ability to enhance learning. Given academic achievement is supported 
by engagement (Finn, 1992; Finn and Voelkl, 1993), and perception and acceptance of 
technology is a prerequisite to students using technology (Lewis, Coursol, & Khan, 2001), 
the current study evaluates the influence of clicker use in the classroom and students' 
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technology proficiency (STP) on students' perceptions of clickers, student engagement, 
and course grade point average. The overall goal of this project is to facilitate a clearer 
examination of the role of technology in the classroom. 

Evidence suggests that students perceive clickers as useful devices in the classroom. Draper 
and Brown (2004) surveyed students to find that using clickers in class allowed students to 
identify content areas they did not fully understand. Importantly, students also found useful 
that instructors could alter their lesson plan "on the fly" if they found through students' 
answers to a question that many students were having difficulty with a specific concept 
covered in class. This idea - coined "contingent teaching" by Draper and Brown (2004) - 
focuses on the role of the instructor as one who responds to the needs of the students 
rather than to the predetermined set of lecture notes. In fact, anecdotal evidence from an 
instructor interviewed by Boyle and Nicol (2003) suggests that "the questioning 
process...gives you more information about where the students are coming from and you 
learn what misunderstandings or misconceptions they have" (p. 52). This process allows 
the instructor to be attentive to student learning, and the student is engaged in the 
instructor's teaching; conceivably an ideal teaching/learning relationship. Thus, it is no 
surprise that some students report more enjoyment in clicker classes relative to standard 
lectures. Indeed, in one study (Marlow, Wash, Chapman, & Dale, 2009) clicker class 
attendance was consistently at 80 to 90%, partly because students enjoyed using clickers. 

Various researchers have sought to discover the specifics behind students' enjoyment of 
clickers in the classroom. Boyle and Nicol (2003) found students in clicker classes felt they 
developed a better understanding of the subject matter relative to standard lectures. 
Students reported having to think more in clicker classes and were able to remember more 
of the information from a clicker class relative to standard lectures. These findings are 
likely driven by the fact that students had to attend the lectures to earn the points available 
through the use of the clicker. Likewise, should students not attend class and, thus, score 
poorly in courses in which clickers are utilized, clickers are likely to be viewed as less helpful 
in the learning process relative to those who are performing well in courses (Trees and 
Jackson, 2007). In other words, course performance may influence students' perceptions 
of the usefulness of a clicker in class. In the present study, we explore the possibility that 
academic performance (course grade) has less to do with clickers than previously 
considered. 

In addition, students' perceptions of clickers are driven by the fact a standard lecture can 
be turned into an interactive pedagogical moment where students can collectively answer 
questions, and get instant feedback. The feedback is unique because clickers can instantly 
aggregate responses, which allows students to stay anonymous, yet contribute to 
discussions on the content discussed during class. Students' ability to answer truthfully 
without embarrassment can instill a sense of ownership over the answer they choose. The 
sense of ownership can be used as a catalyst to improve the quality of student thought 
(Marlow, et al., 2009). Certainly, before the introduction of the clicker into university 
classrooms, instructors still engaged their students through other methods, such as raising 
hands in response to a query. Yet some research has shown that additional engagement 
can come from the clicker class environment. 

For example, Stowell and Nelson (2007) asked students to rate their emotions before, 
during, and after an introductory psychology lecture. During the lecture, students were 
exposed to one of four environments: (1) standard lecture with no review questions, (2) 
review questions requiring a response through raising hands, (3) review questions requiring 
students to hold up response cards, and (4) review questions requiring students to answer 
using their clicker. The most noted finding was the level of honesty reported by the 
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students. Those in the clicker group were more honest in their answers given they were not 
able to socially conform and answer in the way they saw others answer (versus the other 
methods that require commitment before all answers are collected). Thus, while it may 
appear that many students raise their hand and are engaged in a course requiring the 
raising of hands as a response to a query, often that engagement is with their peers as they 
look around the room. The use of clickers allowed the student to engage in the material of 
the class, leaving them to answer what they think is the correct answer and not what other 
students respond as the correct answer. This anonymity is likely appealing to a generation 
of students who are used to communicating while remaining unidentified. Just as a person 
can post comments using a pseudo-name to another's Twitter page, or in a chat room, a 
student can answer using clickers without anyone but that student knowing his response. 

The current generation of students, digital natives or members of Generation NeXt that 
were born between 1981 and the 1990s, who have always known cell phones, computers 
and the Internet (Pew Research Center, 2007), is more likely to interact in class if they can 
use a device to mediate classroom apprehension. 

But what if a student wants to be engaged but is resistant to change, is apprehensive about 
technology affecting his or her grades, or is generally not comfortable with technology? 

Does he or she share in the enjoyment that using clickers in the classroom may bring? 

Lewis, et al. (2001) examined college students' tendencies and comfort level with different 
types of technology. Overall, students felt quite comfortable with e-mail and with general 
Internet use. Their social networking skills allow them to communicate with others in a 
non-threatening environment, one that does not require the student to have their questions 
or comments linked to their face, a prospect students can find intimidating in class. 
Alternately, research has shown that lack of comfort with technology can negatively impact 
students' progress (Zhang & Espinoza, 1998). Computer self-efficacy, or an "individual's 
confidence in his/her ability to use computers" (para. 3) directly influenced the choice to 
perform a given task. It could be, then, that for students to use clickers with ease they 
must have the confidence to do so. We speculate that a student's clicker-use behavior 
(especially for those not so comfortable with technology or resistant to change) is derived 
from the teacher's ability to engage students in the material. There are definite chances that 
technology detracts from learning, regardless of enjoyment. 

In addition to investigating the role of clickers on student engagement, the impact of clicker 
technology on grades is an important question given that grades are a common measure of 
academic performance. Mayer, et al. (2009) exposed students to one of three situations: 

1) a large lecture course in which the instructor did not utilize clickers but did ask at various 
points throughout each class if there were questions, (2) the same large lecture course in 
which the instructor did utilize clickers by asking two to four multiple-choice questions 
during the lecture, or (3) the same large lecture course in which the instructor utilized 
group multiple-choice questioning without the use of clickers. Their results demonstrated 
that students in the clicker section scored significantly higher on exams relative to the 
students who received no questions and those who received group questions. They 
suggest, then, that the use of clickers can improve students' academic performance. 

However, using a similar method, Campbell (2007) showed that students who had higher 
class standing (overall GPA), but were not in clicker classes, outperformed those who were 
in clicker classes, suggesting that the student's overall academic achievement played an 
important role in determining grades. 

Given the apparent conflicting findings about technology in the classroom, the following 
research questions were examined: 
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RQ1: Does student perception of clickers, student engagement, and course GPA 
differ for students who are in clickers classes compared to students that are not in 
clickers classes? 

RQ2: Does student perception of clickers, student engagement, and course GPA 
differ for students who are technologically proficient compared to students who are 
not technologically proficient? 


Method 


Participants 

There were 405 respondents in the study. The participants were students recruited from 
a regional Midwestern university either through classes, or utilizing an online system that 
posts opportunities for students to participate in research. Thirty-three percent were 
freshman (n = 127), 24% were sophomores (n=93), 23% were juniors (n = 87), and 20% 
were seniors and beyond (n = 76). 

A subsample was selected to account for the use of Student Technology Proficiency (STP) 
as an independent variable. Since a requirement of multivariate analysis is that 
independent variables are categorical, the tech savvy data were trimmed to two values: 
low STP and high STP. The lowest 19% of scores (n = 76) on the STP scale signified the 
low proficiency sample, or those with low levels of tech savvyness. Similarly, the top 19% 
of scores (n = 76) were used from the total sample to indicate the high technology 
proficiency sample, or those with high levels of tech savvyness. This procedure reduced the 
sample to 152 participants. Garcia and Zapf (in press) created the valid and reliable 
measure of STP (chronbach's alpha = .82). The current sample had higher reliability 
(chronbach's alpha = .88). Seventy percent of participants (n = 106) were currently in a 
clicker class, and 30% (n = 46) were not. There were more females in the study (n = 103) 
compared to males (n = 48), and 1 non-respondent. Seventy-six percent of participants 
were below 21 years of age, with a mean age of 21 years (SD = 3.65) with values ranging 
from 18 to 46 years. Twenty-eight percent were freshman (n = 43), 26% were sophomores 
(n = 40), 24% were juniors (n = 37), 20% were seniors and beyond (n = 30), and 2 did 
not respond. Multivariate analysis was completed with this smaller sample. 

Materials 

The first independent variable was STP (Garcia and Zapf, in press). The second 
independent variable, student enrollment in a clicker class, was collected using a single item 
in the questionnaire which asked whether the student was currently enrolled in a course 
that used clickers. To measure two key dependent variables - perception of clickers and 
student engagement - it was necessary to create new instruments. The instrument created 
to measure students' perceptions of clickers was developed by modifying 21 existing items 
which were taken from studies relevant to the current research project (Kaleta & Joosten, 
2007; Draper & Brown, 2004; Boyle & Nicole, 2003; Jackson & Trees, 2003). For example, 
the items adapted from Boyle and Nicole's (2003) study referred to "personal response 
systems," but for consistency in the current study this phrase was reworded as "clickers." 
After completing exploratory factor analysis with verimax rotation, the original 21 items 
were reduced to 7 items. Fifty-eight percent of variability was explained with the items. 
Reliability for the seven items was .88. Table 1 displays the final seven items with 
corresponding factor loadings. These were the items used to measure perception of 
clickers. 
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Table 1 . Item Breakdown for Dependent Variables and Component Loadings 


Dependent Variable 


Factor 

Loadings 


Perception of Clickers 


1. Using clickers helps me develop a better understanding of the subject 

matter when compared to traditional lecture based classes. .838 

2. Clickers have been beneficial to my learning. .808 

3. Using clickers helped me to pay attention in class. .780 

4. Using clickers helps me to understand the concepts behind the problems. .820 

5. Clicker questions helped me to know how well I was learning the .676 

material. 

6. Using the clickers helped me get a better grade in this class. .837 

7. I pay attention to whether or not my answer to a clicker question was 

right or wrong. .527 


Student Engagement 


As a result of my involvement in this class, I 

1. Asked questions in class or contributed to class discussions. .945 

2. Discussed ideas from the readings for this class with other students in 

this course. .914 

3. Discussed grades or assignments with your Instructor for this course. .955 

4. Talked about career plans with your Instructor for this course. .957 

5. Discussed ideas from your readings in this class with your Instructor for 

this course. .914 

6. Had serious conversations with students of different perspectives or 

personal values than you. .946 


The student engagement instrument was created by selecting items relevant to the study 
from the reputable National Survey of Student Engagement (NSSE) engagement indicators 
(Kuh, 2001). The NSSE instrument asks students to think about their overall college 
experience, rather than individual classes. In our study, we inquired about individual 
classes. Thus, NSSE questions were used but modified to account for this difference in 
goals. Also, five NSSE indicators were previously validated, however for the current study 
we selected items broadly from the questionnaire for relevance to our research questions, 
and revalidated based on our understanding of engagement in an individual class. For 
example, some items referred to the particular university the student attended (e.g. "to 
what extent does your institution emphasize each of the following."). These questions were 
disregarded. We chose items that reflected engagement in the classroom, and not students' 
overall educational experience. We chose 20 items for the questionnaire. After completing 
an exploratory factor analysis with verimax rotation, the original 20 items were reduced to 
six items. Reliability for the six items was .97. Eighty-eight percent of the variance was 
explained by the six items. Table 1 displays the final six items with corresponding factor 
loadings. These were the items used to measure student engagement in the classroom. 
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Procedure and Design 

The questionnaire measuring STP, perception of clickers, and student engagement was 
administered online to students. All closed-ended questions used a 4-point Likert scale; for 
example, on the engagement scale students could select strongly disagree, disagree, agree, 
or strongly agree. Since we were interested in the educational experience that students had 
in a particular class, we asked participants to only consider one class, and answer all 
questions based on their experiences in that one class. Students were free to think about a 
clicker or non-clicker class. The questionnaire began with items about their perception of 
clickers which were presented in a random order. Next, items about STP were presented, 
followed by engagement items which, again, were all randomly ordered. Finally, 
demographic information was collected, including enrollment in a clicker class or not, year 
in school, gender, and major. Completing the questionnaire took under 10 minutes. 

Course GPA was collected after the semester ended. 

The experiment was a 2 x 2 quasi-experimental design with two independent variables 
(enrollment in clicker class, yes/no; student technology proficiency, high/low), and three 
dependent variables (perception of clickers; student engagement; course GPA). 


Results 

The research questions were analyzed using multivariate analysis of variance (MANOVA). 
The independent variables were enrollment in a clicker class and STP. The dependent 
variables were perception of clickers, student engagement, and course GPA. Bartlett's test 
of sphericity (35.40, df = 18, p < .05) indicated MANOVA was appropriate. Results of 
MANOVA are presented in order of research questions, starting with the effects of student 
enrollment in a clicker class. 

RQ1 investigated the effect of student enrollment in clicker classes versus non-clicker 
classes. Table 2 displays the means and standard deviations for these research questions. 
The multivariate main effect of enrollment in clicker class was not significant, Wilks' lamba 
= .97, F(3,73) = .83, p = ns, power = .22. There were no significant findings at the 
univariate level: perception of clickers, F(l,79) = 1.01, p = ns, power = .17, student 
engagement, F(l,79) = 1.52, p = ns, power = .23, and course GPA, F(l,79)= .21, p = ns, 
power = .07. These findings indicated that a student's involvement in clicker classes versus 
students not enrolled in clicker classes did not appear to change their perception of clickers, 
level of engagement in classroom activities, or student final class grade-point-average. 

Next, RQ2 was explored for differences between high STP and low STP students. Table 2 
contains means and standard deviations for this research question. The multivariate main 
effect of STP was significant, Wilks' lamba = .83, F(3,73) = 5.15, p < .05. There were 
three significant univariate effects: perception of clickers, F(l,79) = 7.08, p < .05, q 2 =.09, 
student engagement, F(l,79) = 4.66, p < .05, q 2 =.06, and course GPA, F(l,79) = 4.14, p 
< .05, q 2 =.05. Planned comparison of difference scores revealed that students who were 
less technological savvy were more engaged than those that were technologically proficient 
(p < .05). Students who were less savvy also had lower favorable perceptions of clickers 
than students who were highly technologically proficient (p < .05). Finally, students who 
were less technologically savvy had higher average course GPAs than students who were 
more technologically proficient (p < .05). In general, whether or not a student was enrolled 
in a clicker class had no impact on students' perception of clickers, student engagement, or 
course GPA, whether a student was low in STP or high in STP had significant impacts on all 
three dependent variables. 
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Table 2. Means and Standard Deviations for Dependent Variables 



Clicker Class 

Technoloaical 

Dependent Variables 



Proficiency* 


Yes 

No 

High 

Low 


M (SD) 

M (SD) 

M (SD) 

M (SD) 

Student Engagement 

2.33 (.28) 

2.26 (.26) 

2.27 (.31) 

2.36 (.23) 

Perception of Clickers 

3.22 (.51) 

2.95 (.65) 

3.22 (.54) 

2.88 (.53) 

Course GPA 

2.98 (1.12) 

2.94 (1.12) 

2.67 (1.24) 

3.31 (.83) 


* All findings were significant at p < .05 between high/low Technological Proficiency on the three dependent 
variables. 


Discussion 

The purpose of the current study was to separate the influences of clickers from STP to 
improve our understanding of the effect of clickers in college classrooms and therefore to 
examine the belief that using technology in the classroom is beneficial, regardless of 
pedagogy or student abilities. The research questions asked whether clickers themselves 
would have an influence on students' perception of clickers, classroom engagement, and 
academic performance, or whether STP might instead (or in addition to) play an important 
role in students' perception of clickers, classroom engagement, and academic performance. 
Contrary to previous research, students in clicker classes did not show increased favor 
towards clickers (Patry, 2009), higher engagement levels (Trees & Jackson, 2007; Beekes, 
2006), or higher course grades compared to students who were not in clicker classes 
(Kaleta & Joosten, 2007). Our research found that STP did influence the dependent 
variables, and our interpretation of the findings support what classically trained teachers 
likely already suspect: technology alone does not guarantee learning. 

On the findings regarding perception of clickers, there was no difference reported between 
those students enrolled in a clicker class and those not enrolled in a clicker class; yet 
students who were higher in STP had more favorable perceptions of the clickers than 
students low in STP. This could be due to a mix of high and low STP students in the 
classroom; no noticeable difference appeared between those enrolled and not enrolled in 
clicker classes because of STP characteristics that were present in all classes from which 
the sample was taken. 

A second finding revealed that students who were low in STP were significantly more 
engaged than those high in STP. It could be that the difference in students' comfort level 
with technology influenced their engagement, regardless of their clicker class involvement. 
Not surprisingly, the higher STP students thought more favorably about the clicker's 
educational value likely because they were already comfortable with technology and, thus, 
"wired" to think the devices would aid their learning. 

The final dependent variable, course grade point average, again demonstrated no significant 
different between students in clicker and non-clicker classes. Some researchers would 
argue that this could be because learning happens through classroom engagement (Finn, 
1992; Finn and Voelkl, 1993), or that the educational value of technology only occurs when 
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students are comfortable and knowledgeable with technology (Lewis, 2001). The current 
findings support a different view of clickers. Students low in STP earned higher grades; 
moreover, although these low STP students had less favorable perception of clickers, they 
were also more engaged in the classroom. Certainly the current study supports previous 
research that engagement is related to students' grades, given low STP students were more 
engaged and earned higher grades. Yet, comfort with and knowledge of clickers (high STP 
students) did not seem to improve engagement in the classroom or course grade. Marlow 
et al. (2009) investigated clickers use in four distinct disciplines. Their major conclusions 
support our findings; students perceived clickers to be excellent drivers of participation, 
but had doubts about the clickers' potential to increase learning. 

As an example of how all three of the explored dependent variables interact, consider the 
following scenario: students low on STP may make more of an effort to stay engaged in 
ways not related to the use of clickers or technology (and throughout the semester 
increased their computer self-efficacy). In this way, there may be a tight connection 
between engagement and grades, rather than high expectations of clickers and grades. 
Given it was the low STP students who were more engaged and the low STP students who 
earned higher grades, it is likely that a critical element is the teacher/student interaction; 
one that should not be ignored or pushed aside. "Contingent teaching" is a crucial facet of 
using technology in the classroom, and is related more to effective teaching practices, 
rather than the technology itself. For example, Lantz (2010) prescribes a specific method 
for helping instructors use clickers to teach specific concepts. Students are stimulated by 
effective teaching practices, not clickers directly, but the students who recognize this 
connection the most are probably those who already pursue higher academic achievement. 

Thus, unlike Mayer, et al. (2009), the current study has shown that using clickers in the 
classroom was not enough to bolster performance in a course. Could students high in STP 
rely too heavily on the use of clickers in the classroom, especially given their high 
expectations for their usefulness? In this way, then, these students may not utilize 
additional study methods, whereas the student who is low in STP, and does not have high 
expectations for the role of clickers in their learning, continues traditional study methods 
and, therefore, may earn a higher grade. Regardless, the crucial point for instructors is 
that classroom technology is related to student performance in the classroom, but perhaps 
in unexpected inverse relationships (e.g. highly tech savvy students may indeed do poorly 
with all else held constant). We should not overlook the possibility that some students may 
stick with courses simply because they think clickers are exciting, yet not give too much 
thought to their impact on the final grade. 

There are three directions for future research thus far unexplored. First, it is necessary to 
further scrutinize clickers with qualitative research. Our path is to triangulate the current 
findings by conducting focus groups comprised of students from each of the four controlled 
conditions in this study. For example, we speculate that high STP students may be 
motivated to take clicker courses that are rich in technology use, but we do not know if 
there are other motivations, or whether other factors are influencing our dependent 
variables. Focus groups are specifically useful because they afford participants 
opportunities to explore and come to conclusions collectively; and thus we can attain a 
richer data set. Porter and Tousman's (2010) research on nursing students' perceptions 
of clickers is an example of statistical and qualitative inquiry. Second, new studies should 
investigate the combined effects of the teacher's methods with student technology 
proficiency. Nicolle and Lou's (2008) considered the factors that influence faculty to adopt 
technology. Evidence suggests that collegiality affects adoption of technology. We wonder 
whether factors that influence faculty to use technology and students' own comfort level can 
influence student learning. Third, additional research should investigate appropriate 
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methods for using clickers. In this study we suggest that using clickers helps teachers 
accomplish "contingent teaching" (Draper & Brown, 2004), an approach implemented by 
Porter and Tousman (2010) with "question-driven instruction," but we do not test for 
methods of instruction with clickers. 

The current research was originally motivated by the curiosity to determine whether clickers 
were useful in classrooms for educational purposes. We found that while clickers may be 
useful, they should be used with some degree of caution. As instructors, we must balance 
our desire to teach with technology because we think it could improve the learning 
experience, with the knowledge that technology is not a panacea for students doing well. 
Simply because students are savvy with technology, and because they have a positive 
perception about clickers, does not equate to higher grades. For those instructors that do 
use clickers, it is wise to dedicate time in class to discuss why clickers are beneficial, and 
then students can decide if they want to come along for the ride. The literature points to 
many valid scenarios where clickers can play a supporting role to an instructor's pedagogical 
practices, such as classroom enjoyment (which may lead to engagement; Kaleta and 
Joosten, 2007). At the end of the day, however, relative to today's modern cell phones and 
the many sophisticated networking devices that have infiltrated our culture, clickers are still 
very simple devices that, too, have their limits. 
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